System and method for performing production line product identification

ABSTRACT

In an illustrative embodiment, a system for identifying products on a production line includes image capturing devices that acquire images of containers moving along a production line at an inspection location. The system also includes a rejection device and a controller that configures the image capturing devices for image acquisition based on properties of the containers, identifies a product associated with each of the containers based on a portion of a product identification code and a portion of additional features detected in the images, and determines whether the identified product matches predetermined properties or characteristics, resulting in a pass result, otherwise a non-pass result occurs. When a non-pass result occurs, the controller outputs a signal to actuate the rejection device that removes the container from the production line.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/498,267 entitled “System and Method for Performing Production LineProduct Identification,” filed Apr. 26, 2017. This applicationincorporates by reference, in its entirety, the following prior patent:U.S. Pat. No. 9,147,326, entitled “Encoder Based Speed CompensatedReject System and Method,” filed Jan. 28, 2013. All above identifiedapplications are hereby incorporated by reference in their entireties.

BACKGROUND

The present disclosure relates to a system and method for identifyingproducts in a manufacturing production line.

Foods, medications, dietary supplements and other packaged productsprocessed in a manufacturing facility are typically controlled to ensureconsumer safety. The dangers in manufactured products include variousforms of contamination and distribution of incorrect product. In somecases, when contamination of a product is identified, that particularproduct may have to be recalled and/or removed from production in orderto ensure safety of those who consume the product. In addition,sometimes mismatched products that do not correspond to the productbeing processed on a production or packaging line may get mixed up withthe product being processed or packaged, which can lead to errors inproduct deliveries to consumers.

In some cases, manufacturers can use computerized systems to identifythe products on the production line based on an identification code suchas a Universal Product Code (UPC). However, such computerized systemsmay rely on capturing a full UPC code or other identification markingsin order to positively identify the product. In some instances where theproducts being processed on a manufacturing line have a uniform shape(e.g., rectangular) and orientation relative to a UPC detection devicemay be relatively uniform. However, a single manufacturing line in aproduction facility may process many types of products having a varietyof packaging shapes that can result in various aspects and orientationsof the packaging being presented to the UPC detection device, which canresult in a partial UPC or no UPC being presented to the UPC detectiondevice.

SUMMARY OF ILLUSTRATIVE EMBODIMENTS

The forgoing general description of the illustrative implementations andthe following detailed description thereof are merely exemplary aspectsof the teachings of this disclosure, and are not restrictive.

In certain embodiments, a system for identifying products on aproduction line includes a conveyor that transports containers ofproducts along a production line that includes an inspection locationincluding multiple cameras configured to obtain images of the containersat the inspection location from various positions and orientationangles. A controller receives the images captured by the cameras at theinspection location and detect features of a label on the packaging ofthe containers that can include at least a portion of a barcode, brandand product logos, ingredient lists, or nutrition information. If thefull barcode is not visible in the images captured by the camera, thecontroller can identify the product associated with a given containerbased on the features that are visible in the images. In addition, theproduct can be identified based on the detected features in the imageseven if no portion of the barcode is present. The controller calculatesa quality score for the images obtained by each of the cameras and usesthe image with the highest quality score to identify the product basedon stored product pattern data. If the detected features match thepattern data for the product that is currently being processed on theproduction line, then the item is allowed to pass. If, however, thedetected features do not match the pattern data, either due to amismatch condition where an incorrect item is identified or a no matchcondition where neither the correct item nor an incorrect item areidentified, then the controller may actuate a rejection device thatdiverts the item away from the production line. In some implementations,a predetermined number of items having a no match condition are allowedto pass on the production line before actuation of the rejection device.

Benefits of the system include being able to identify products on highvolume production lines where the containers may be touching each other,causing portions or all of the barcode and label to be obscured fromview of the cameras. In addition, containers having cylindrical orrounded shapes can still be identified by the system even if a portionor all of the barcode to be obscured from view of one or more of thecameras. Using a combination of features to identify the productsimproves the accuracy and speed of product detection and allows theproducts to be identified even if a barcode is not captured in theimages by the cameras.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate one or more embodiments and,together with the description, explain these embodiments. Theaccompanying drawings have not necessarily been drawn to scale. Anyvalues dimensions illustrated in the accompanying graphs and figures arefor illustration purposes only and may or may not represent actual orpreferred values or dimensions. Where applicable, some or all featuresmay not be illustrated to assist in the description of underlyingfeatures. In the drawings:

FIG. 1 illustrates an exemplary product identification system;

FIG. 2 illustrates an exemplary block diagram of a programmable logiccontroller (PLC);

FIG. 3 illustrates an exemplary block diagram of types of data andprocesses executed by the PLC;

FIG. 4 illustrates an exemplary label for an item;

FIGS. 5A-5B illustrate an exemplary flow diagrams of a productidentification process; and

FIG. 6 illustrates an exemplary flow diagram of a system configurationprocess.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The description set forth below in connection with the appended drawingsis intended to be a description of various, illustrative embodiments ofthe disclosed subject matter. Specific features and functionalities aredescribed in connection with each illustrative embodiment; however, itwill be apparent to those skilled in the art that the disclosedembodiments may be practiced without each of those specific features andfunctionalities.

Reference throughout the specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with an embodiment is included inat least one embodiment of the subject matter disclosed. Thus, theappearance of the phrases “in one embodiment” or “in an embodiment” invarious places throughout the specification is not necessarily referringto the same embodiment. Further, the particular features, structures orcharacteristics may be combined in any suitable manner in one or moreembodiments. Further, it is intended that embodiments of the disclosedsubject matter cover modifications and variations thereof.

It must be noted that, as used in the specification and the appendedclaims, the singular forms “a,” “an,” and “the” include plural referentsunless the context expressly dictates otherwise. That is, unlessexpressly specified otherwise, as used herein the words “a,” “an,”“the,” and the like carry the meaning of “one or more.” Additionally, itis to be understood that terms such as “left,” “right,” “top,” “bottom,”“front,” “rear,” “side,” “height,” “length,” “width,” “upper,” “lower,”“interior,” “exterior,” “inner,” “outer,” and the like that may be usedherein merely describe points of reference and do not necessarily limitembodiments of the present disclosure to any particular orientation orconfiguration. Furthermore, terms such as “first,” “second,” “third,”etc., merely identify one of a number of portions, components, steps,operations, functions, and/or points of reference as disclosed herein,and likewise do not necessarily limit embodiments of the presentdisclosure to any particular configuration or orientation.

Furthermore, the terms “approximately,” “about,” “proximate,” “minorvariation,” and similar terms generally refer to ranges that include theidentified value within a margin of 20%, 10% or preferably 5% in certainembodiments, and any values therebetween.

All of the functionalities described in connection with one embodimentare intended to be applicable to the additional embodiments describedbelow except where expressly stated or where the feature or function isincompatible with the additional embodiments. For example, where a givenfeature or function is expressly described in connection with oneembodiment but not expressly mentioned in connection with an alternativeembodiment, it should be understood that the inventors intend that thatfeature or function may be deployed, utilized or implemented inconnection with the alternative embodiment unless the feature orfunction is incompatible with the alternative embodiment.

Aspects of the present disclosure are directed to identifying productson a manufacturing line that can be configured to process a wide varietyof products for manufacturing and packaging. When items such as foods,medications, dietary supplements and similar products are manufacturedand packaged, it may be necessary to inspect the products (or theirpackaging/container) to ensure that the correct product is packaged,shipped and, ultimately, used or consumed by the purchasers of theproducts. When an incorrect item is identified in the production orpackaging process, it may be appropriate to reject the incorrect itemby, for example, ejecting or diverting the item from the productionline. In some implementations, cameras are configured on themanufacturing line to capture images of the products that are used topositively identify the products based on a detected barcode or otherunique identification code.

Manufacturing production lines are very often configured to process awide variety of products that come in a wide variety of shapes andsizes, which makes positively identifying the products based on abarcode alone more difficult because the high density of products on theproduction line along with the shape of product packaging may obscuresome or all of the barcode from view. In the implementations describedfurther herein, the system can determine whether all of a barcode for aproduct is visible in a captured image of a product on the productionline. If the full barcode is not visible then a captured image or ifnone of the barcode is present in the captured image, then the systemcan use a combination of features detected in images of the product toidentify the product. Using a combination of barcode and featuredetection to identify products on the production line improves accuracyrates of product identification and also improves overall processingefficiency since the system does not have to wait for an image of a fullbarcode to identify the product.

FIG. 1 illustrates a system 100 for identifying products according tocertain embodiments of the present disclosure. The system 100 may beused to identify products (such as items 110, for example) transportedby conveyor 105 past at least one predetermined inspection location. Insome implementations, the items 110 may be associated with a particularproduct that is being packaged for shipment to a customer on amanufacturing line. For example, inspection of products (or productpackaging/containers) may take place in a manufacturing process as anin-line operation while, in other embodiments, inspection of productsmay take place during packaging or boxing of finished products. In someimplementations, the items 110 may be identical packages of a type offood, drug, or other type of product. Although only five items 110 areillustrated in FIG. 1, it can be understood that a typical productionline processes a continuous stream of items like those illustrated asitems 110. The system 100 may be configured to identify the items 110 toensure that the correct products have been processed and packaged for acustomer. In some implementations, the items 110 may be packaged inboxes, jars, tubes, tubs, or any type of container that can have avariety of shapes and sizes. As illustrated in FIG. 1, conveyor 105transports the items 110 from left to right.

At the inspection location (the approximate position of item 110C), thesystem 100 may include one or more cameras 115 positioned to captureimages or video of the items as they pass the predetermined inspectionlocation on conveyor 105. Although three cameras are shown in FIG. 1,some implementations may include additional cameras, while otherembodiments may only have one or two cameras. The number of cameras, insome examples, may be determined based on one or more of a number ofsides of the product packaging, number of sides of the product packagingincluding identifying information, size of the product, density of theproduct on the conveyor 105 (e.g., if more than 1 item may beside-by-side), etc. For example, some product items may not have usefulidentification information on a top surface, so camera 115C may not beused to identify such items.

In some implementations, the number of cameras 115 included in thesystem 100 may be based on feature capture angle ranges of each of thecameras 115 for the items 110 that are processed by the system 100. Inaddition, the system 100 can activate and/or deactivate one or morecameras 115 of the system 100 based on the feature capture angle ranges.The feature capture angle ranges may correspond to a range of rotationaldegrees that each camera is able to capture a feature for a particularitem 110, which can be based on characteristics of the cameras 115(e.g., camera orientation/aspect angle, distance of camera from theitem, etc.) as well as dimensions of the packaging of the items 110 andfeatures of a label on an outer surface that is used to identify theitems 110.

For example, the items 110 can include a cylindrically shaped packagesuch as a soup can, condiment jar, yogurt container, or single servingcereal bowl that may have a label affixed to one or more outer surfacesof the items 110 that distinguish the items 110 from other products. Inone implementation, the label is made of a flexible material (e.g.,paper, flexible plastic) that is wrapped around an outer rounded surfaceof the items 110. In other implementations, the label may be printed orstamped on the outer surface of the items 110. The label for the items110 may include a number of identifying features that can be used by thesystem 100 to distinguish the items 110 from other products. Theidentifying features can include a barcode 140, matrix or 2D barcode, QRcode, or any type of identification code that distinguishes the item 110from other types of products such as a universal product code (UPC),brand and/or product logos, ingredient lists, nutrition information,quantity information, and any other information included on the label.In circumstances involving cylindrical or rounded shape packagingcausing partial obscuring of an affixed label, the cameras 115 may beable to capture a particular feature on the label for a predeterminednumber of degrees of rotation of the item 110.

In some implementations, the number of cameras 115 that are used toidentify the items is based on how many degrees of rotation a particularcamera 115 is able to capture a full or complete barcode 140 on thelabel. As the items 110 are transported by the conveyor 105, arotational orientation of the items 110 with respect to each of thecameras 115 may be (or become) essentially random due to a randomizednature of how the items 110 are placed on the conveyor, contact betweenthe items 110 on the conveyor 105 that causes rotational shifting of theitems 110, and the like. For example, items 110A-110C are shown in avariety of different rotational orientations with respect to the cameras115. In some embodiments, such as those exemplified in FIG. 1, thespacing of items on conveyor 105 may be minimized to achieve a highproduction throughput. In some embodiments, the production throughput,for example, may exceed 500 items per minute (<120 ms per item). In someembodiments, the items on conveyor 105 may be touching or in very closeproximity to one another. Under such conditions, the barcodes 140 onsome items 110 (such as item 110C, for example) may be hidden from viewof the any of the cameras 115. For example, only a part of barcode 145on item 110C may be visible to any of the cameras 115.

In one example where the system 100 includes four cameras 115, eachcamera may capture the full barcode 140 for the each of the items 110over 70 degrees of rotation of the individual item 110. Therefore, thecameras 115 can be positioned to capture the full barcode of the items110 over 280 degrees of rotation of the item 110, and the capturedbarcode 140 can be used by the system 100 to identify the item 110. Forthe remaining 80 degrees of rotation, the cameras 115 can capture apartial barcode and/or other features of the label affixed to thepackaging of the item 110, which can be used by the system 110 toidentify the item 110. Because the types of items 110 processed by thesystem 100 have packaging shapes and sizes, label features, andbarcodes, each type of item 110 may have different ranges over which thebarcode 140 and other features may be captured. For example, anothertype of product, such as a yogurt container, may have a barcode that isrotated 90 degrees from the barcode 140 shown on the items 110 inFIG. 1. The rotated barcode may result in a greater range of orientationangles over which a full barcode may be captured by the cameras 115,which can result in a lower number of cameras being used by the system100 to capture images used to identify the items 110. In addition, otherdesign characteristics of the labels of the items 110 may affect thenumber of cameras 115 that are used to identify the product. Forexample, the graphical features of a single serving cereal bowl labelmay be more easily identified in the captured images than the graphicalfeatures of a soup can label, which can result in only two cameras 115being configured to capture images when single serving cereal bowls arebeing processed by the system 100 while a greater number of cameras 115may be used to capture images of the soup cans.

In some implementations, the system 100 can adaptively configure thenumber of cameras 115 that are activated to capture images of the items110 based on characteristics of the labels affixed to the items 110being transported by the conveyor 105. By adaptively determining thenumber of cameras that are used by the system 100, power savings can beachieved by powering down the cameras 115 that are not in use or puttingthe cameras 115 that are not in use into a sleep mode. In addition,using fewer cameras 115 to acquire images of the items 110 reduces aprocessing load on the PLC 120 or other processors associated with thesystem 100, which can cause a greater number of items to be processed bythe system 100 in a shorter period of time. Because of the highproduction throughput (>500 items per minute) through the system 100, insome embodiments, the process of identifying a product may be requiredto be completed in less than, for example, 120 ms. Therefore, reducingthe number of acquired images for processing can result in improvedprocessing times. In addition, the system 100 can automatically controlan orientation angle and position of the cameras 115 based on the numberof cameras 115 being used. The use of the term “camera” herein is notintended to be limiting. Any type of area or line scan image capturedevice, including still cameras, video cameras, line scan cameras andthe like may be used as an application demands.

In some implementations, the system logs a timestamp of item informationcapture. The system 100 may determine, for example, that an item 110 hasreached the inspection location based on a speed of conveyor 105 and atime at which the item 110 is detected by sensor 150.

The images captured by cameras 115 may be fed into programmable logiccontroller (PLC) 120 that executes one or more software processesassociated with the system 100 and outputs control signals to othercontrollers and electronically-activated components of the system 100.One example of a suitable PLC for the system 100 is an Allen BradleyPLC. As an illustrative example, FIG. 2 provides a simplified hardwareblock diagram of a PLC 200, which can be implemented as the PLC 120 inthe system 100. The description of the PLC 200 is not meant to belimiting, and can include other components than those described herein.The PLC 200 may include a central processing unit (CPU) 206 thatexecutes one or more software processes associated with the system 100.Software instructions for the processes can be stored in memory 202. Thememory 202 can include both volatile and non-volatile memory and canstore various types of data associated with executing the processesrelated to capturing images of items 110 on the conveyor and identifyingthe product based on the captured images. The PLC 200 includes an inputinterface 204 for connecting to various devices that provide inputs tothe PLC 200 such as the cameras 115, conveyor 105, rejection conveyor145, conveyor controller 135, sensor 150, rejection device 130, and anyother device associated with the system 100. The PLC 200 also includesan output interface for connecting and providing control signals to anydevice controlled by the PLC 200 including the cameras 115, rejectionconveyor 145, conveyor controller 135, rejection device 130, and anyother device controlled by the PLC 200. The PLC 200 also includes apower supply 210 as well as a communication device 212 that allows thePLC 200 a wired or wireless communication interface between the PLC 200and any other device of the system 100 or other systems.

Referring back to FIG. 1, in some implementations, the PLC 120 canoutput data and control signals to conveyor controller 125, whichcontrols operation of conveyor 105 and rejection line conveyor 145,which can include starting and/or stopping conveyors 105 or 145 oradjusting the speed or direction of the conveyors 105 or 145. Thefunctions and operations performed by the conveyor controller 135 canalso be integrated into functions performed by the PLC 120.

The PLC 120 may also control the acquisition of images by cameras 115by, for example, controlling a precise time for image capture. The PLC120 may also control other parameters of cameras 115 such as focus,aperture, white balance, exposure/shutter speed, etc. The PLC 120 canalso power down (or put into sleep mode) any cameras that are not beingused to capture images of a particular item 110 or activate andconfigure for use any additional cameras. In some embodiments, sensor150 may provide an advance indication for a times at which a productwill reach the inspection location. In other embodiments, cameras 115may capture a continuous sequence of images (a video sequence, forexample), and the PLC 120 may select the most appropriate image for eachitem as it passes the inspection location. In some embodiments, sensor150 may be an optical sensor that may employ a light emitter and photosensor that detects the presence of an item on the conveyor 105. Inother embodiments, sensor 150 may be an ultrasonic sensor that mayemploy an ultrasonic emitter and ultrasonic sensor that detects thepresence of an item on the conveyor 105. In yet other embodiments,sensor 150 may employ two devices (a light emitter and photo sensor, forexample), one on either side of conveyor 105, to sense the passing of anitem. In another embodiment, the products on conveyor 105 may includeradio frequency identification (RFID) tags. In such an embodiment,sensor 150 may include RFID detection circuitry.

In some implementations, the PLC 120 configures one or more of thecameras to acquire images of the items 110 based on the type of productbeing processed by the system 100. For example, the PLC 120 can storevarious types of data in memory that can be used to determine how toconfigure the cameras 115 to acquire images of a particular product. Thedata that is used to determine the number, position, and orientation ofthe cameras 115 of the system can be determined based on storedpackaging pattern data that can include label templates for varioustypes of products processed by the system 100, product identificationdata that includes unique identification codes for each of the itemsthat may correspond to the UPC or another type of unique identifier,production schedule data that includes times that various types ofproducts are scheduled to be processed by the system 100, and cameraconfiguration data that indicates a current status of each of thecameras 115 with respect to position, orientation angle, and the like.

The PLC 120 may process images from cameras 115 to identify indicia orunique features on each item 110 that correspond to the packagingpattern data to positively identify the item 110 whose image is capturedby the cameras 115 and determine whether the item 110 corresponds to atype of item currently being processed. In some implementations, the PLC120 determines a quality score for each of the images captured by thecameras 115. The quality score provides an indication of how well thefeatures in a captured image can be used to positively identify the item110. The quality score may correspond to a value in any range of values,such as 0 to 10, 0 to 100, 0 to 1000, or any other range.

In some implementations, if a calculated quality score is greater than apredetermined threshold, then the PLC 120 may determine that a capturedimage corresponds to a particular product. In addition, images havingquality scores less than a second predetermined threshold may beconsidered as having too few features to be able to accurately recognizea type of product and thus any image having a quality score less thanthe second predetermined threshold is discarded. The PLC 120 may alsoidentify the item 110 based on a relative quality score that provides anindication of how well the captured image represents a particularproduct versus any other product that is processed by the system 100.For example, an item 110 with a low relative quality score can indicatethat the features detected in the image indicate correspondence to twoor more different products within a predetermined range of similarity.On the other hand, an item having a high relative quality score canindicate that the features detected in the image are strongly associatedwith one product as opposed to any other product processed by the system100.

In addition, the quality score for each image may be represented byeither a percentage or raw score. As a percentage, the quality scoreindicates what percentage of the image includes detectable features ofthe item 110 that can be used to distinguish the item 110 from otherproducts that are processed by the system 100. In other examples, thepercentage quality score indicates a percentage of the container for theitem 110 that has been captured by the image. For example, an imagewhere an entire container of the image 110 is captured may have a higherpercentage quality score than an image where only a portion of thecontainer is captured. In implementations where the quality score isrepresented by a raw score, the score may represent a number ofdetectable features of the item 110 that can be used to distinguish theitem 110 from other products that are processed by the system 100.

In some examples, if a full barcode 140 is captured in an image, thatparticular image receives a highest possible quality score in the givenrange. In examples where the captured images include a partial barcodeor no barcode, the quality score may be calculated to indicate a numberor percentage of features that are detected in the image that can beused to identify the item 110. For example, the detected features caninclude full or partial product or company logos, nutrition information,ingredient list, quantity information, or any other image features. Insome implementations, the product can be positively identified based onthe detected features even if no barcode is present in the image. Oncethe quality scores have been determined for all of the captured imagesof an item 110, the PLC 120 selects the image with the highest qualityscore to be compared to packaging patterns of products stored in memoryof the PLC 120. The PLC 120 may use any type of image processing and/orpattern recognition algorithm that is known to compare the image withthe highest quality score to the stored packaging patterns. For example,the PLC 120 can use edge detection, background subtraction, or framedifferencing algorithms to locate the items 110 within the image.Because the cameras 115 capture images of the items 110 in a relativelystatic environment with consistent lighting, less robust objectdetection can be used to locate the items 110 within the image. Forexample, frame differencing provides highly accurate object detectionresults in a relatively static environment with low processing time.Meanwhile, template matching object detection algorithms provide highlyaccurate results but also may have high processing times. In someexamples, the processing time for template matching can be reduced byidentifying selected features/indicia of a pattern template for aproduct that distinguish a given product from other products processedby the system and comparing the selected features to detected featuresin the images captured by the cameras 115. In some implementations, theselected features can include particular color patterns at variouslocations in the image, detection of various shape patterns in theimages, and alignment of the various components of an item label withinan image. The templates that are compared to the items 110 detected inthe images may be represented by eigenvectors that can be compared tovector representations of the item features or portions of item featuresdetected in the images. If the identified product for the item 110matches the product that is currently being processed by the system 100,resulting in a match condition, then the item 110 is allowed to continueon the conveyor 105. If, however, a match condition does not occur foran item 110, then the item may be diverted onto rejection conveyor 145by rejection device 130.

In some examples, when a match condition does not occur, then the item110 may have been matched to another product that does not correspond tothe product currently being processed by the system, resulting in amismatch condition. In some aspects, if a mismatch condition isdetected, then the mismatched item is diverted onto the rejectionconveyor 145 by the rejection device 130, and a lockout of the system100 may occur in which the conveyor 105 is shut down until a lockoutclearing process is performed. In some implementations, the lockoutclearing process may include authentication of an authorized user with aRFID key and/or password who clears the lockout and restarts the system100 after investigation into the cause of the mismatch condition. Inother examples, if a match condition does not occur, then the item 110may not have been matched to the correct product nor an incorrectproduct, resulting in a no match condition. The no match condition mayoccur in situations where the images captured by the cameras 115 do notinclude enough detectable features to positively identify the item 110.The no match condition may also occur when one or more of the cameras115 may have malfunctioned or failed. In some examples, a predeterminednumber of items 110 may be allowed to pass on the conveyor 105 beforethe non-matched item is diverted onto the rejection conveyor 145 by therejection device 130 and/or the system lockout occurs.

The system 100 may also include a rejection device 130 and correspondingrejection conveyor 145 that may be configured to eject or divert aspecific item from conveyor 105 under the control of controller 125. Insome implementations, if the PLC 120 determines that the item 110Eidentified on the conveyor 105 does not correspond to the type of itemcurrently being processed, the PLC 120 can issue a control signal to therejection device 130 to divert the item 110E onto the rejection conveyor145 and away from the path provided by the conveyor 105. The rejectiondevice 130 can be any type of device that can either directly makecontact or cause another object to make contact with the item 110E inorder to cause the item 110E to move off the conveyor 105 and onto therejection conveyor 145. In some implementations, the rejection device130 may be an arm, blade, pusher, stepper motor, servo motor, aircylinder, pneumatic blow-off, or other device. The type of rejectiondevice 130 that is used to divert the item 110E off the conveyor 105 maybe based on a type of packaging of the item 110E. For example, gentlerrejection devices such as the stepper motor, servo motor, or pusher maybe used to divert breakable or deformable items that may be packaged inglass or another type of breakable or highly malleable material.Similarly, lighter items that are unbreakable may be blown off theconveyor 105 and onto the rejection conveyor 145 with the air cylinderor pneumatic blow-off. In some implementations, the PLC 120 can alsoactivate an alarm or other type of alert mechanism when an item isdiverted onto the rejection conveyor 145 in response to a detection of aproduct mismatch or no match condition. The alarm can include any typeof audio or visual indication that alerts a user that a product mismatchor no match condition was detected, and the corresponding item wasdiverted off the conveyor 105 and onto the rejection conveyor 145.Examples of alarms include any combination of an audible tone or spokenalarming condition message, a blinking light or other type of visualindication, and a product message output report to a computer log ormessaging system that can provide text messages, emails, or other typesof messages to system users indicating that a product mismatch or nomatch condition was detected. In some examples, when a predeterminednumber of product mismatches or no matches are detected within apredetermined period of time, the PLC 120 may shutdown the conveyor 105until a user verifies that the correct products are being processed bythe system 100 and resets the PLC 120 to clear the alarming condition.

The system 100 illustrated in FIG. 1 provides a simplified arrangementof a product inspection system according to some embodiments of thepresent disclosure for the sake of clarity. Other components that may beuseful for some embodiments described in the present disclosure are notillustrated in FIG. 1. These include, but are not limited to, lighting,support structures, conveyor drive, camera control equipment, etc. Forexample, to capture appropriate images of the bar code 140 or otheridentifying indicia, the lighting may be adjusted to avoidglare/reflection/oversaturation of the image, etc.

Turning to FIG. 3, an alternative data processing and storageconfiguration for the system 100 is presented. In some implementations,the software processes associated with the system 100 can be executed byprocessing resources such as servers, database servers, or any othertype of processing resources including cloud-based computing resources.For example, processing system 300 is associated with and connected tothe system 100 and can include one or more engines or modules thatperform processes associated with identifying products transported bythe conveyor 105 of the system 100. In some implementations, theprocessing system 300 can connect to the system 100 via a wired orwireless connection. References to the engines or modules throughout thedisclosure are meant to refer to software processes executed bycircuitry of one or more processing circuits, which can also be referredto interchangeably as processing circuitry. The processes executed bythe processing system 300 correspond to processes that may be executedby the PLC 120 of the system 100 described above in FIG. 1.

For example, the processing system 300 includes a camera configurationengine 302 that configures the cameras 115 of the system 100 based onthe type of product being processed by the system 100. In someimplementations, the camera configuration engine 302 can adaptivelyconfigure the number of cameras 115 that are activated to capture imagesof the items 110 based on characteristics of the labels affixed to theitems 110 being transported by the conveyor 105. For example, the cameraconfiguration engine 302 can activate and/or deactivate one or morecameras 115 of the system 100 based on the feature capture angle ranges.The feature capture angle ranges may correspond to a range of rotationaldegrees that each camera is able to capture a feature for a particularitem 110, which can be based on characteristics of the cameras 115(e.g., camera orientation/aspect angle, distance of camera from theitem, etc.) as well as dimensions of the packaging of the items 110 andfeatures of a label on an outer surface that is used to identify theitems 110. In one example where the system 100 includes four cameras115, each camera may capture the full barcode 140 for the each of theitems 110 over 70 degrees of rotation of the individual item 110.Therefore, the cameras 115 can be positioned to capture the full barcodeof the items 110 over 280 degrees of rotation of the item 110, and thecaptured barcode 140 can be used by the system 100 to identify the item110. For the remaining 80 degrees of rotation, the cameras 115 cancapture a partial barcode and/or other features of the label affixed tothe packaging of the item 110, which can be used by the system 110 toidentify the item 110 even when some or none of the barcode is presentin the captured images. The camera configuration engine 302 may alsocontrol other parameters of cameras 115 such as focus, aperture, whitebalance, exposure/shutter speed, etc. The camera configuration engine302 can also power down (or put into sleep mode) any cameras that arenot being used to capture images of a particular item 110.

The processing system 300 also includes an image acquisition engine 304that acquires the images captured by the cameras 115. In someimplementations, the image acquisition engine 304 outputs a controlsignal to the cameras 115 to capture one or more images of the items 110based on sensor data received from sensor 150 indicating a position ofthe items 110 on the conveyor 105. The image acquisition engine 304receives the images from each of the cameras 115 and annotates capturedimage data 322 for each of the images with the particular camera 115that captured the image along with configuration parameters for thatparticular camera 115.

The processing system 300 also includes an image processing engine 306that detects features of the images that can be used to identify thetype of product being transported on the conveyor 105. In someimplementations, the features can include any identifying marks that arevisible on an outer surface of the products and may include a barcode,brand and/or product logos, ingredient lists, nutrition information,quantity information, and any other information included on the label.The image processing engine 306 can use any image processing algorithmsthat are known to detect the features of the images. The features caninclude patterns, colors, sizes, shapes, and locations of variousobjects that are detected on the label of the product.

The processing system 300 also includes a score determination engine 308that calculates a quality score for each of the captured images. Thequality score provides an indication of how well the features in acaptured image can be used to positively identify the item 110. Thequality score may correspond to a value in any range of values, such as0 to 10, 0 to 100, 0 to 1000, or any other range. In some examples, if afull barcode 140 is captured in an image, that particular image receivesa highest possible score in the range. In examples where the capturedimages include a partial barcode or no barcode, the quality score may becalculated to indicate a number or percentage of features that aredetected in the image that can be used to identify the item 110. Forexample, the detected features can include full or partial product orcompany logos, nutrition information, ingredient list, quantityinformation, or any other image features. Once the quality scores havebeen determined for all of the captured images of an item 110, the scoredetermination engine 308 selects the image with the highest qualityscore to be compared to packaging pattern data 314 stored in datarepository 322.

The processing system 300 also includes a product comparison engine 312that compares the image with the highest quality score to the packagingpattern data 314 for all possible products that are manufactured on aproduction line or at a manufacturing facility. The product comparisonengine 312 may use any type of image processing and/or patternrecognition algorithm that is known to compare the image with thehighest quality score to the packaging pattern data 314. In someimplementations, key feature points are identified in the capturedimage, which are compared to key feature points in the stored packagingpatterns. If the identified product for the item 110 matches the productthat is currently being processed by the system 100, then the item 110is allowed to continue on the conveyor 105. Degree of certainty inmatch, for example, may be required to be at least 95%, 99%, or 100%.If, however, the identified product does not correspond to the productcurrently being processed by the system 100, then the item is divertedonto rejection conveyor 145 by rejection device 130. For example, ifdegree of certainty in match is less than a threshold amount, such asless than 95% certainty, the item may be rejected or classified as anon-match to a particular product. Conversely, if the system generates apositive match (e.g., meeting or exceeding a threshold of certainty) fora different product than the anticipated product, corresponding to amismatch condition, the system 100 may divert the item onto therejection conveyor 145.

The processing system 300 also includes a rejection engine 312 thatcontrols the diversion of rejected items onto the rejection conveyor 145in response to detection of a product mismatch condition or no matchcondition by the product comparison engine 310. The system 100 may alsoinclude a rejection device 130 and corresponding rejection conveyor 145that may be configured to eject or divert a specific item from conveyor105 under the control of controller 125. In some implementations, if theproduct comparison engine 310 determines that an item identified on theconveyor 105 does not correspond to the type of item currently beingprocessed, the rejection engine 312 can issue a control signal to therejection device 130 to divert the item onto the rejection conveyor 145and away from the path provided by the conveyor 105. The rejectiondevice 130 can be any type of device that can either directly makecontact or cause another object to make contact with the rejected itemin order to cause the item to move off the conveyor 105 and onto therejection conveyor 145. In some implementations, the rejection device130 may be an arm, blade, pusher, stepper motor, servo motor, aircylinder, pneumatic blow-off, or other device. In some implementations,the rejection engine 312 can also activate an alarm or other type ofalert mechanism when an item is diverted onto the rejection conveyor 145in response to a detection of a product mismatch or no match condition.

Further, in some embodiments, the rejection engine 312 may confirmredirection of the product onto the rejection conveyor 145. Anadditional sensor, for example, may identify that a new item has beenadded to the rejection conveyor 145.

In some implementations, the processing system 300 is connected to datarepository 322 via a wired or wireless connection. The data repository322 can be configured to store data that is used by the processingengines of the processing system 300 or produced by the processingsystem 300 during execution of the processes. For example, the datarepository 322 can include packaging pattern data 314 that can includelabel templates for various types of products processed by the system100, product identification data 316 that includes unique identificationcodes for each of the items that may correspond to the UPC or anothertype of unique identifier, production schedule data 318 that includestimes that various types of products are scheduled to be processed bythe system 100, and camera configuration data 320 that indicates acurrent status of each of the cameras 115 with respect to position,orientation angle, and the like. The data repository 322 can alsoinclude captured image data 322 that can be used as historical data tofurther refine the packaging pattern data 314 for the products processedby the system 100.

Turning to FIG. 4, an exemplary label 400 for an item is illustrated. Inimplementations where the item has a rounded or cylindrical shape, thelabel 400 may be made of a flexible material (e.g., paper, flexibleplastic) that is wrapped around an outer rounded surface of the item. Inother implementations, the label 400 may be printed or stamped on theouter surface of the item. The label 400 for the item may include anumber of identifying features that can be used by the system 100 todistinguish the item from other products. For example, the features caninclude a brand logo 402, product logo 404, product picture 406,nutrition information 408, ingredients 410, and barcode 412. In someimplementations, the system 100 can identify the item based on thedetected features as well as the locations of the features relative toother features on the label 400. Additional features other than thoseshown in FIG. 4 can also be included on the label 400, which may also beused to identify the product, such as inspection stickers, quantityinformation, cooking directions, etc.

FIGS. 5A and 5B are flow diagrams that schematically illustrate a method500 of product identification, and processing of the results of theidentification, according to some embodiments of the present disclosure.The descriptions for the flow diagrams illustrate a particular orderingof processes, steps, and decision points, but it can be understood thatthe steps of the processes can be performed in a different order.Additionally, certain steps of the flow diagrams, in other embodiments,may be performed in parallel.

In some implementations, the method 500 begins with configuring asystem, such as the system 100 of FIG. 1, for processing a particularproduct (502). Turning to FIG. 6, a flow diagram of a method 600 forconfiguring a system is illustrated. Turning to FIG. 6, in someimplementations, the method 600 begins with receiving identificationdata for the product (602) from a sensor (e.g., 150), one or more ofcameras (e.g., cameras 115), or a user input indicating that items ofthe product are being transported by a conveyor (e.g., 105). Theidentification data can include a UPC or other unique identificationcode or machine readable indicia, product name, or any other informationthat can be used to identify the product. In some embodiments, thesystem 100 can use stored production schedule data to determine the typeof product be transported on the conveyor 105.

In response to receiving the product identification data for theproduct, in some implementations, a number of cameras to activate isdetermined based on feature capture angle ranges for the product (604).The feature capture angle ranges may correspond to a range of rotationaldegrees that each camera is able to capture a feature for a particularitem, which can be based on characteristics of the cameras (e.g., cameraorientation/aspect angle, distance of camera from the item, etc.) aswell as dimensions of the packaging of the items and features of a labelon an outer surface that is used to identify the items.

If the number of cameras to activate to obtain full coverage of thelabel on the items by the cameras is less than the total number ofcameras connected to the system (608), in some implementations, thecameras to be configured to capture images of the items are selected(610). For example, the PLC 120 of FIG. 1 may select the cameras. Theselection, for example, can be based on an orientation of the cameraswith respect to one another as well as the characteristics of thepackaging and label of the items. The selected cameras may beconfigured, for example, by outputting control signals to turn on orwake the cameras from a sleep mode if necessary and to modify anorientation angle and/or position of the cameras so that the cameras cancapture images that allow for positive identification of the items bythe captured images.

Referring back to FIG. 5A, in response to the system being configuredfor product processing, in some implementations, each of the camerascaptures at least one image of an item as the items pass a predeterminedinspection location on a conveyor (504). The PLC 120 of FIG. 1, forexample, may determine that an item 110 has reached the inspectionlocation based on a speed of conveyor 105 and a time at which the item110 was detected by the sensor 150.

For each captured image, in some implementations, it is determinedwhether a full barcode is visible in the image (506). The PLC 120 ofFIG. 1, for example, may determine whether the complete barcode iscaptured within an image. If the full barcode is visible in the image,in some implementations, the barcode is extracted from the image (510).However, if the full barcode is not visible in the image, in someimplementations, an image feature pattern is extracted from the image(508). In some embodiments, the image feature pattern includes partialor no barcode as well as partial or full product or company logos,nutrition information, ingredient list, quantity information, color,product shape, information region alignment, or any other imagefeatures.

In some implementations, an image quality score is determined based onthe detected barcode or image feature pattern (512). For example, thePLC 120 of FIG. 1 may determine a quality score for each of the imagescaptured by the cameras 115. The quality score provides an indication ofhow well the features in a captured image can be used to positivelyidentify the item 110. The quality score may correspond to a value inany range of values, such as 0 to 10, 0 to 100, 0 to 1000, or any otherrange. In some examples, if a full barcode 140 is captured in an image,that particular image receives a highest possible score in the range. Inexamples where the captured images include a partial barcode or nobarcode, the quality score may be calculated to indicate a number orpercentage of features that are detected in the image that can be usedto identify the item 110. For example, the detected features can includefull or partial product or company logos, nutrition information,ingredient list, quantity information, or any other image features.

In some implementations, images of an item are processed until all ofthe images captured by each of the cameras have been processed (514).For example, the PLC 120 of FIG. 1 may continue to process imagesobtained by each of the cameras and determine quality scores of eachimage. In some embodiments, a number of images are processedsimultaneously in parallel, which reduces a total processing time. Insome embodiments, if a quality score matching a particular threshold isreached (e.g., 98/100, 995/1000, etc.) the high quality image may beselected and the remaining images discarded from further processing. Forexample, to reduce processing cycles and increase speed, upon obtainingan image determined capable of producing a match, the system may ceaseprocessing remaining images captured by the cameras.

In some implementations, processing of images may be done on acamera-by-camera (or camera angle by camera angle) basis starting with acamera most likely to capture a positive match. For example, anglescapturing sides of packages rather than an upper view of packages may beconsidered more likely to produce matching product data.

Turning to FIG. 5B, once the quality scores have been determined for allof the captured images of an item (or, conversely, one image or athreshold number of images deemed containing a threshold quality ofimage are identified), in some implementations, the image with thehighest quality score may be identified and selected for comparison topackaging patterns of known products (516). For example, the PLC 120 ofFIG. 1 may identify and select one or more images for comparing topackaging patterns stored in memory of the PLC 120.

In some implementations, image processing and/or pattern recognitionalgorithm(s) are applied to compare the image(s) with the highestquality score(s) to the stored packaging patterns (518). The PLC 120 ofFIG. 1, for example, may compare one or more images to stored packagingpatterns for all types of products that are processed by the system 100or in the same manufacturing facility as the system 100. In otherembodiments, only packaging patterns associated with the anticipateditem are used to identify or discard the item. Because a probability ofdetecting a product mismatch may be less than a probability of detectingthe correct item on the conveyor 105, the PLC 120 may prioritizecomparing the highest scoring image to the packaging pattern for theproduct being processed by the system 100 so that a match is detectedbefore comparisons are performed with packaging patterns for otherproducts. In some embodiments, key feature points are identified in thecaptured image, which are compared to key feature points in the storedpackaging patterns. In some implementations, a match condition isdetected when a number or percentage of key feature points detected inthe image that correspond to key feature points of a stored productpattern for the product being processed by system 100 is greater than apredetermined threshold, otherwise, a mismatch condition or a no matchcondition is detected. If, however, the number or percentage of keyfeature points that correspond between the captured image and a storedproduct pattern is less than the threshold but greater than the numberor percentage of corresponding feature points for any of the otherstored product patterns by more than a predetermined amount, the PLC 120may still detect a match condition.

If the identified product for the item matches the product that iscurrently being processed by the system (520), in some embodiments, theitem is allowed to continue on the conveyor (522). For example, the PLC120 of FIG. 1 may determine that a particular item 110 may continue onthe conveyor 105. If, however, the identified product does notcorrespond to the product currently being processed by the system, thenin some examples, it is determined whether a mismatch condition or a nomatch condition exists.

A mismatch condition occurs when the item is positively identified asanother product which is different from the product which is currentlybeing processed by the system (521). If, in some aspects, the mismatchcondition has occurred, then the item is diverted onto a rejectionconveyor by a rejection device (524). For example, the conveyorcontroller 135 of FIG. 1 may divert the particular item 110 onto therejection conveyor 145. In the event a mismatch condition is detectedthe PLC may halt the production line and prevent restarting of theproduction line until a user logs in (authenticates to) the PLC andclears the lockout code.

A no match condition occurs when the container cannot be positivelyidentified. This occurs, for example, when a product label cannot beproperly read or when the product label can be read but does not matchany product in the PLC memory. In some examples, in the event of a nomatch condition the conveyor controller 135 of FIG. 1 may divert theparticular item 110 onto the rejection conveyor 145. The PLC 120 maykeep a running count of “no match” events and when the number ofdetected no match conditions is greater than a predetermined threshold(523) then the item production line may be halted as described above.However, if the number of detected no match conditions is less than thepredetermined threshold, then the PLC 120 of FIG. 1 may simply rejectthe particular item and allow the production line to continue operating.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the present disclosures. Indeed, the novel methods, apparatusesand systems described herein can be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods, apparatuses and systems described herein can bemade without departing from the spirit of the present disclosures. Theaccompanying claims and their equivalents are intended to cover suchforms or modifications as would fall within the scope and spirit of thepresent disclosures.

1. (canceled)
 2. A system for inspecting containers on a continuouslymoving conveyor, the system comprising: a plurality of image acquisitiondevices positioned proximate an inspection location on a continuouslymoving conveyor, wherein each of the plurality of image acquisitiondevices is adjustably oriented to acquire images of a plurality ofcontainers moving along the conveyor from a different acquisition angle;and a controller communicatively coupled to the plurality of imageacquisition devices, wherein the controller comprises processingcircuitry, and a non-transitory computer readable memory coupled to theprocessing circuitry, the memory storing machine-executableinstructions, wherein the machine-executable instructions, when executedon the processing circuitry, cause the processing circuitry to, adjust,based on a first type of a plurality of types of containers of therespective container, one or more settings for the plurality of imageacquisition devices, for each of the plurality of containers, identify,from a plurality of images of the respective container, an image forcontainer identification, wherein the image for container identificationincludes a first number of identifiable features that is higher than asecond number of identifiable features in other of the plurality ofimages of the respective container, determine, based on a comparisonbetween the identifiable features in the image for containeridentification with packaging pattern data for the plurality of types ofcontainers stored in memory, whether a match condition exists betweenthe respective container and the first type of container, and responsiveto determining that the match condition does not exist, cause actuationof a rejection device to remove the container from the conveyor.
 3. Thesystem of claim 2, wherein the identifiable features of the respectivecontainer comprise at least one of a portion of a product identificationcode and a portion of additional features of the respective container.4. The system of claim 3, wherein the additional features of therespective container comprise at least one of a brand logo, a productlogo, an ingredient list, nutrition information, and quantityinformation for the respective container.
 5. The system of claim 3,wherein the product identification code comprises at least one of abarcode, a QR code, and a universal product code (UPC).
 6. The system ofclaim 2, wherein adjusting the one or more settings of the plurality ofimage acquisition devices comprises adjusting the respective acquisitionangle of a portion of the plurality of image acquisition devices suchthat the plurality of image acquisition devices are configured tocapture an entire outer surface of the respective container at theinspection location.
 7. The system of claim 2, wherein adjusting the oneor more settings for the plurality of image acquisition devices is basedon at least one of a size, shape, label dimensions, and labeldistribution for the first type of container.
 8. The system of claim 2,wherein adjusting the one or more settings for the plurality of imageacquisition devices comprises deactivating a portion of the plurality ofimage acquisition devices.
 9. The system of claim 2, wherein the one ormore settings for the plurality of image acquisition devices comprise atleast one of an orientation aspect angle and a position of each of theplurality of image acquisition devices.
 10. The system of claim 2,wherein identifying the image for container identification comprisesidentifying a subset of the plurality of images of the respectivecontainer containing one or more of the identifiable features of therespective container.
 11. The system of claim 10, wherein identifyingthe image for container identification comprises identifying the imagefor container identification from the subset of images, wherein theimage for container identification includes the first number ofidentifiable features that is higher than a third number of identifiablefeatures in each image of the subset of images.
 12. The system of claim2, wherein causing actuation of the rejection device comprises causingactuation of the rejection device to divert the respective item from theconveyor to a rejection conveyor.
 13. The system of claim 2, wherein therespective container has a cylindrical shape.
 14. The system of claim 2,wherein determining that the match condition does not exist comprisesdetermining that a no-match condition or a mismatch condition existsbetween the respective container and the packaging pattern data for thefirst type of container.
 15. The system of claim 14, wherein determiningthat the mismatch condition exists comprises detecting, based on thecomparison of the identifiable features of the image for containeridentification with the packaging pattern data for the plurality oftypes of containers, that the identifiable features in the image forcontainer identification matches product packaging data for another typeof the plurality of types of containers than the first type ofcontainer.
 16. The system of claim 15, wherein the machine-executableinstructions, when executed on the processing circuitry, further causethe processing circuitry to: determine whether a predetermined number ofoccurrences of the mismatch condition have been detected; and responsiveto determining the predetermined number of occurrences of the mismatchconditions have been detected, causing the conveyor to halt.
 17. Thesystem of claim 14, wherein determining that the no-match conditionexists comprises detecting, based on the comparison of the identifiablefeatures of the image for container identification with the packagingpattern data for the plurality of types of containers, that theidentifiable features in the image for container identification fail tomatch the product packaging data for any of the plurality of types ofcontainers.
 18. A method comprising: determining, by processingcircuitry of a controller, a first type of item of a plurality of typesof items for processing on a conveyor; adjusting, by the processingcircuitry, one or more settings for each of a plurality of imageacquisition devices for capturing images of a plurality of the firsttype of items at an inspection location on the conveyor, wherein theadjusting of the plurality of image acquisition devices configures theplurality of image acquisition devices to capture images an entire outersurface of the plurality of the first type of items; acquiring, by theplurality of image acquisition devices, images of the plurality of thefirst type of items passing the inspection location of the conveyor; bythe processing circuitry for each item of the plurality of the firsttype of items, identifying, from a plurality of images of the respectiveitem, an image for item identification, wherein the image for itemidentification includes a first number of identifiable features that ishigher than a second number of identifiable features in other of theplurality of images of the respective item; determining, based on acomparison between the identifiable features in the image for itemidentification with packaging pattern data for the plurality of types ofitems stored in memory of the controller, whether a match conditionexists between the respective item and the first type of item; andresponsive to determining that the match condition does not exist, causeactuation of a rejection device to remove the item from the conveyor.19. The method of claim 18, wherein the identifiable features of therespective item comprise at least one of a portion of a productidentification code and a portion of additional features of therespective item.
 20. The method of claim 18, wherein determining thatthe match condition does not exist comprises determining that a no-matchcondition or a mismatch condition exists between the respectivecontainer and the packaging pattern data for the first type of item. 21.The method of claim 20, wherein determining that the mismatch conditionexists comprises detecting, based on the comparison of the identifiablefeatures of the image for item identification with the packaging patterndata for the plurality of types of items, that the identifiable featuresin the image for item identification matches product packaging data foranother type of the plurality of types of items than the first type ofitem.